A Multi-Modal Approach for Early Autism Recognition Using Deep Learning and Sensory Data

Abstract

With Early detection of Autism Spectrum Disorder (ASD) can make a life-changing difference in a child’s journey, helping them receive the right support at the right time. This project introduces a smart, hybrid system that combines advanced deep learning technology with proven treatment methods, aiming to close the gap between diagnosis and meaningful help. It examines various types of information such as behavioural assessments and sensory response patterns to train a model that can identify early signs of autism with high accuracy and consistency. When the system detects a possible case, it provides structured, theory-based activities designed to develop cognitive, social, emotional, and communication skills in young children. These activities are based on widely accepted approaches and are intended to encourage steady developmental growth. A major strength of this system is its automated reporting feature, which gathers diagnostic insights, structured treatment recommendations, and predicted progress into a clear, easy-to-read report for parents, therapists, and healthcare professionals, ensuring everyone stays informed and aligned. By blending advanced computational analysis with trusted treatment practices, the system ensures both accurate detection and a smooth path to intervention. It supports early diagnosis, ongoing guidance, and continuous monitoring, helping reduce delays and improving engagement in a child’s developmental plan. This combined approach demonstrates how technology and professional expertise can work together to create an accessible, practical tool for ASD management. Its goal is to transform early detection into immediate, meaningful action that nurtures potential, builds confidence, and helps shape a brighter future for every child.

Introduction

ChatGPT said:

Summary

The proposed M-CARE (Autism Behavioural Screening and Therapy System) is a web-based platform designed to support the early screening and intervention of Autism Spectrum Disorder (ASD). It combines behavioural analysis with machine learning and multimodal data from video and audio to estimate autism risk.

The system captures behavioural indicators such as eye contact, facial expressions, head movements, and voice characteristics during a structured screening session. These features are processed by a machine-learning model to generate an autism risk score and risk level. Based on the results, the system produces a downloadable screening report and may recommend suitable therapy activities.

The literature survey highlights that traditional autism screening depends heavily on questionnaires, clinical interviews, and expert observation, which may not be easily accessible to all families. Recent research has explored computer vision, facial-expression analysis, eye-gaze detection, and speech processing for automated autism screening. However, many existing approaches focus on either visual or audio information alone. The proposed system addresses this limitation through a hybrid/multimodal approach.

The system follows a three-layer architecture:

• Presentation Layer: Provides registration, profiles, screening sessions, prompts, and result displays.
• Application Layer: Handles authentication, real-time communication, feature extraction, machine-learning prediction, report generation, and therapy recommendations.
• Data Layer: Securely stores user details, screening sessions, extracted features, therapy records, and results.

The system also includes a Therapy Centre, where parents can upload reports, generate age-based therapy plans, participate in interactive therapy sessions, and monitor progress through weekly or monthly graphs.

The results indicate that the multimodal approach improves prediction accuracy compared with a single-model approach and can identify relevant facial and behavioural characteristics associated with autism risk. Overall, the system aims to increase early awareness, provide accessible screening support, and guide parents toward appropriate professional evaluation and therapy. Importantly, it is intended as an assistive screening tool and not a replacement for professional medical diagnosis.

Conclusion

This project presents a practical and user-friendly system for early Autism Spectrum Disorder (ASD) risk assessment and therapy support. The system combines behavioural observation through video and audio analysis with machine learning to estimate autism risk in a structured manner. By guiding users through a screening session and automatically generating a detailed report, the platform simplifies the initial evaluation process and makes it more accessible. In addition to screening, the system provides therapy suggestions and progress tracking features, helping parents and caregivers take the next step if a higher risk level is identified. The integration of screening, report generation, and therapy support within a single platform makes the system organized and easy to use. Although it is not a substitute for professional medical diagnosis, it acts as an assistive tool that promotes early awareness and timely intervention. Overall, the project demonstrates how technology can be effectively applied to support early developmental assessment and structured therapeutic guidance.

References

[1] Wang, P., & Zhou, Y. (2021). Early Autism Detection Using Multimodal Sensor Data and Deep Learning. [2] Kumar, N., & Pandey, S. (2022). A Review on Early Detection of Autism Using Artificial Intelligence Techniques. [3] Hasan, M. T., Rahman, A., & Islam, S. (2020). Sensory Data-Based Autism Diagnosis Using Random Forest. [4] Duan, J., Li, L., Zhang, Q., Qin, J., & Zhou, Y. (2023). AI-Assisted Behavioural Analysis for Autism Recognition. [5] Ahmed, J., & Ali, S. (2021). Hybrid Deep Learning Model for Autism Spectrum Disorder Detection. [6] Kaliyadan, F., Sharma, R., & Mehta, P. (2019). A Machine Learning Approach for Early Detection of Autism Spectrum Disorder.

Copyright

Copyright © 2026 Shaikh Ahmed, Shaikh Faeik, Shaikh Israhil, Prof. Sonali Karthik. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.